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ABSTRACT 

The utilization of progreun evaluation may be made 
more effective by means of the application of contemporary persuasion 
theory. The Elaboration Likelihood Model — a model of cognitive 
processing, ability, and motivation — was used in this study to test 
the persuasive effects of source credibility and involvement on 
message acceptamce of evaluation reports. The study used a tvo-'by-two 
design to investigate the effects of the two independent variables of 
ascribed credibility and participant issue involvement. Subjects were 
63 graduate students in four education classes. A simulated report 
describing a 5-year pilot study of a career options program 
supposedly implemented in seven school districts in a southern state 
was reviewed by ths subjects. A cover letter contained the 
experimental manipulation, incorporating the variables of source 
credibility and issue involvement. An 11-point Likert scale was used 
to assess the effectiveness of the manipulation of source credibility 
and issue involvement. Results indicate that the credibility of an 
' evaluator is an important aspect that educators attend to when 
reading evaluation reports. The study provided preliminary analysis 
for judging the psychometric soundness of items used in a subsequent 
investigation utilizing high-level administrators and certified 
program evaluator i as subjects. A 34-item list of references ie 
included. (TJH) 
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ABSTRACT 

The utilization of program evaluation may be made more 
effective with the use of persuasion theory. The Elaboration 
Likelihood Model was employed to test the persuasive effects 
credibility and involvement. The results indicate that the 
credibility of an evaluator is an important aspect that 
educators att^d to when reading evaluation reports. 



THE EFFECT OF PERSUASION ON THE UTILIZATION 
OF PROGRAM EVALUATION INFORMATION: 
A PRELIMINARY 9TUDY 

As the importance of program evaluation has increased, 
problems related to utilization of the evaluation reports have 
been more widely recognized. During the 1960 's, program 
evaluations were thought to provide information for action 
(Weiss, 1972). Evaluators expected that their reports would 
lead to major program change when problems were detected. 
Evaluators became concerned when their efforts resulted in 
little direct utilization (Alkin, Daillak, & White, 1979, pp. 
14-16; Thompson & King, 1981, pp. 3-4). As a consequence, 
researchers began to question whether these action expectancies 
were realistic. They also sought to determine if the use and 
non-use of evaluations by decision makers had discernible 
patterns. What emerged was a new and expanded definition of 
utilization and of how utilization related to the complexities 
of organizational decisions. 

Among the first to acknowledge a problem with the action 
concept were Patton et al. (1977), who noted that "evaluation 
literature has focused on too narrow a definition of evaluation 
research impacts'* (p. 161). The perception of underutilization 
ultimately led evaluatio«*^ theorists to redefine utilization 
(Brown, Newman, & Rivers, 1984; Leviton & Hughes, 1981). Two 
major distinctive definitions for program evaluation 
utilization surfaced. Program utilization is viewed from both 
a narrow and a broad perspective. If evaluation information is 



meant for iininediate and direct use in improving the quality of 
a program, then use of the narrow perspective is indicated. 
However, when evaluation findings are used as supplemental 
information for future decisions, the broader perspective of 
use is more appropriate. Additional research from the late 
1970 's also indicates that the broader perspective is becoming 
more commonly used by evaluators (Alkin, Daillak, & White, 
1979; Knorr, 1977; Patton, 1978). with the expanding 
definitions, utilization becomes "less dramatic and more 
difficult to explicitly measure and demonstrate. It represents 
a view of evaluation in which the role of human interaction in 
the communication process is given more credance" (Brown & 
Braskamp, 1980, p. viii). 

Researchers (Leviton & Hughes, 1981, p. 528), concerned 
about program evaluation utilization, have systematically 
differentiated saveral catagories of utilization: 
instrumental, conceptual, and persuasive. Instrumental use is 
an ideal. In instrumental use an evaluation is completed and 
direct discernible changes occur. Conceptual use of research 
information was defined by Rich (1977) as "influencing a 
policymaker's thinking about an issue without putting 
information to any specific, documentable use" (p. 200) . 

Persuasive use "draws on evaluation evidence in attempts 
to convince others to support a political position, or to 
defend such a position from attack" (Leviton & Hughes, 1981, p. 
528). But as Patton (1978) states, "The traditional academic 
values of many social scientists lead them to want to be 



nonpolitical in their research. Yet they always want to affect 
government decisions. The evidence is that they cannot have it 
both ways" (p. 46) . 

Evaluators have a responsibility to provide not only o 
comprehensive quality evaluation but the additional 
responsibility of increasing the utilization of evaluation. So 
more evaluators have now come to appreciate House's (1977, p. 
5) position that one of the primary functions of evaluations is 
to persuade. For example, following a five-year series of 
studies, Newman, Brown and Braskamp (1980) argued that the 
"evaluation reporting process can be viewed as analogous to a 
persuasive communication message" (p. 29). Thompson (1981), in 
his review of communication theory and program evaluation, 
emphasized applying theoretically grounded persuasion 
principles to establish a possible relationship between 
persuasive effort and program evaluation utilization. 

The purpose of the present study was to investigate the 
effects of source credibility and issue involvement on message 
acceptance of evaluation reports. The present study provided 
preliminary analysis for judging the psychometric soundness of 
items employed in a subsequent investigation (Eason & Thompson, 
1988) utilizing high-level administrators and certified program 
evaluators as subjects. Both studies have been completed. 
This paper reports the results from pilot study subjects, i.e., 
graduate education students at a large university. The study 
was grounded in contemporary persuasion theory, and the 
experimental variations examined were selected on this basis. 



Contemporary Persuasion Theory 
Pour primary theoretical approaches comprise contemporary 
persuasion theory. These include learning, consistency, 
perceptual, and functional approaches (Shelby, 1986, p. 10). 
The first approach, learning theory, attempts to explain or 
predict the relationship between a stimulus and a response 
(Staats, 1967, p. 373). The stimulus may be the source, the 
message, or the channel. The response is how the receiver 
reacts to the stimulus. 

Several persuasion models have developed from learning 
theory. Two of the most commonly used are the information 
processing and cognitive response models (Shelby, 1986, p. 11). 
Information processing models developed by Hovland and 
associates received much attention in the 1950 "s and 1960 'Sr \ 
more recent theory, the cognitive response model developed by 
Brock (1967) and Greenwald (1968) and refined by Petty (1977), 
focuses on the receiver's cognitive response to a message. The 
theory hypothesizes Lhat thoughts generated by the initial 
persuasive communication influence the receiver's attitude 
concerning acceptance or rejection of the message. The 
persistence of attitude is a ^unction of the elaboration 
created by the cognitive responses to the message. The 
principles of this theory provided one important framework for 
the present study. 

Consistency theory, a second approach to persuasion, 
focuses on the relationship between the stimulus and the 
receiver's frame of reference. In contrast to learning theory. 
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consistency theory focuses on the stimulus and the response. 
How much a receiver has stored in memory about the message 
content and/or the source affects the perceived persuasive 
intent • 

Dissonance^ a construct of consistency theory, has been 
applied to persuasion theory. Dissonance involves the 
inconsistency found to exist when new environmental phenomena 
confront previously formed, internally consistent knowledge, 
opinions, and attitudes of a person (Festinger, 1957). When 
dissonance occurs, tension or discomfort develops which will 
then motivate the person to achieve consistency or consonance. 
For example, cognitive dissonance occurs when a person who 
smokes is shown a graphic film of the health perils of smoking 
To relieve tension, the person rationalizes that quitting 
smoking leads to weight gain which is also hazardous to health 
Smoking is continued and the person has achieved consonance 
with the habit. 

The third major theoretical approach to persuasion is the 
perceptual approach. It emphasizes how receivers perceive the 
message. The perceptual approach focuses in particular upon 
the receiver's attitude or frame of reference about the 
message. The approach emphasizes that it is the perception of 
the message rather than the actual message that affects 
attitude change. 

Fourth, the functional approach to persuasion states that 
potential persuasive effects must be relevant to the needs of 
the receiver (Katz, 1960). Katz hypothesized that attitudes 
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serve at least one of the following functions: adjustment, 
ego-defense, value-expression, and knowledge acquisition (p. 
461). An important theory in the functional approach, 
compliance-gaining, assumes that power is necessary for 
persuasion to occur (Carlsmith, Collins & Helmrieich, 1966, p. 
333) . 

The Elaboration Likelihood Model 
The diversity of data on the traditional source, message, 
receiver, and channel variables, and the various theoretical 
approaches to persuasion led Petty and Cacioppo (1986) to 
develop a general theory of attitude change called the 
Elaboration Likelihood Model (ELM). The ELM attempts to 
integrate the many conflicting findings and theories from 
previous work into one viable framework. 

"Elaboration" is defined as the degree to which a person 
cognitively processes the message. "Likelihood" refers to the 
person's ability and motivation. Therefore, if a person has 
the ability and is motivated co process issue-relevant 
information, elaboration likelihood is high. Conversely, if 
ability and motivation are not present, elaboration likelihood 
is low. 

The ELM divides the previous empirical findings and 
theories into two "routes to persuasion" (Petty & Cacioppo, 
1986, p. 3): central and peripheral. Within the central route 
a receiver is led by ability and motivation to cognitively 
consider the issues relevant to the message. The theorists 
hypothesize that the effort involved in mentally processing 



information makes persuasion through the central route more 
permanent and more resistant to change. A central route is 
involved when a voter critically analyzes the political 
platform of a candidate rather than the candidate's projected 
charisma • 

Conversely, in the peripheral route a receiver is led to 
react to a simple cue, rather than by cognitively processing 
issues. The credibility of the source, the topic of the 
communication, the attractiveness of the source, the length of 
the message, and the media type are only a few types of cues. 
In the previous example, the peripheral route might involve 
voters watching a TV commercial and focusing on the candidate 
shown with a happy family. 

One of the basic differences between central and 
peripheral routes is the permanency of their effects on 
attitude change. Peripheral cues are easier to process, but 
because they lack cognitive press they are less resistant to 
competing messages. However, the poi^er of cues can lead to 
persuasion even in the absence of issue-relevant information. 
For example, an automobile dealer may ignore the relative 
quality of a product but still persuade the buyer with the 
simple message, "Buy American." Conversely, central arguments 
are difficult to establish but do result in longer lasting 
attitudinal change. The ELM predicts that when both the source 
and message contain combinations of central issue-relevant 
arguments and peripheral cues, the power to persuade is 
increased • 
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The ELM also indicates that the relative effectiveness of 
persuasion techniques is, in part, a function of the 
characteristics of the receiver. Receivers may be 
differentiated by bias, motivation, prior knowledge, locus of 
control, and intelligence. Additionally, different receivers 
have different needs for information and different kinds of 
decisions to make (Newman, Bull, Brown, & Rivers, 1986). 

Literature on the comparison of administrator and 
evaluator views of evaluation is scarce. Thompson and Miller 
(1984) explored these two perceptions of evaluation by 
employing Meltsner's model. The research question was whether 
administrators and evaluators had similar views of evaluation 
and the evaluator's role. Two of the Q-technique factors 
consisted of a mix of both groups. The results suggest that 
some administrators and evaluators hold similar views of 
evaluation. Also, since most administrators grouped in the 
first two subject clusters, the results suggest that 
administrators are more homogeneous in their views of program 
evaluations. The views of the evaluators were not as 
consistent. Presumably evaluators and administrators can 
communicate more adequately if they understand each other's 
viev's of program evaluation. 

Two persuasive elements were the primary focus in the 
present study. These were source credibility and issue 
involvement, with respect to source credibility, attribution 
theory suggests that source characteristics have an impact on 
message acceptance or rejection. The source factor can affect 
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persuasion in several different ways: serving as arguments, 
serving as cues^ or affecting the extent or direction of issue 
and argument processing (Pecty & Cacioppo, 1986^ p. 47). 

The source factor can affect attitude change under varying 
conditions of high or low elaboration likelihood. When a 
person is unmotivated anJ unable to evaluate message arguments, 
simple cues such as source credibility are relied upon. Thus, 
under low elaboration likelihood a highly credible source tends 
to enhance persuasion. But when an audience member is highly 
motivated and able to cognitively process the issue-relevant 
arguments, the effects of source credibility tend to disappear. 
These principles have been corroborated in experiments 1 
research (Maddux & Rogers, 1980; Pallak & Francia, 1985). 

A second variable, issue involvements is a motivational 
variable affecting elaboration likelihood (Petty & Cacioppo, 
1986, p. 81). Issue involvement, also referred to as personal 
relevance, concerns the extent to which the topic of the 
message is of relevance to the receiver. High personal 
relevance of a policy change in school dress code occurs if the 
change takes place in a receiver's own school. Low personal 
relevance might be typified by a change caking place in another 
state. 

Involvement is based on social judgment theory which 
posits latitudes of acceptance, rejection, or noncommitment . 
After reviewing the persuasion research, Petty and Cacioppo 
(1986, p. 82) found that few researchers had considered the 
effects of issue-relevant arguments on persuasion. For 
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example, issue relevance of a new exam policy might be high for 
some undergraduate students but the topic of import duties on 
raw silk might be less relevant to these students. 

If the informat' is consistent with the subject's 
initial •>pinion, processing of the strengths of the message 
will occur. However, if the information is inconsistent with 
the subject's opinion, high relevance topics will generate 
counterarguments to the message. Consequently, Petty and 
Cacioppo (1986) incorporated personal relevance as a central 
principle to their ELM. As personal relevance increases, 
people become more motivated to work harder to process 
issue-relevant arguments (Petty & Cacioppo, 1979a, p. 1915). 
The more involved the receiver is in the outcome, the more 
resistant the receiver will also be to changing attitudes. 
Again, previous research supports the^ s theoretical principles 
(Norton, 1986; Petty & Cacioppo, 1979b). 

In summary, the literature portrays persuasion theory as 
a viable framework for investigation regarding 'e effective 
program evaluation utilization. Several studies have been 
reported involving persuasion in studies of program evaluation 
utilization. For example, series of studies conducted by 
Brown and Braskamp (1980) presented evidence suggesting that 
persuasion theory provides a viable framework for studying 
program evaluation utilization. 

The present study employed a 2 x 2 (credibility 
manipulation x involvement manipulation) design to investigate 
the effects of the two independent variables on ascribed 
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evaluator credibility and participant issue involvement. 
Specifically, the study was designed to address two research 
questions • First, will peripheral involvement persuasive 
effects occur even if evaluation information does not directly 
and immediately impact evaluation report readers? Second, will 
evaluator peripheral credibility cue effects occur 
independently of involvement effects? 

Method 

Subjects 

The subjects in the present study were graduate stddents 
(n « 63) in four education classes. Evaluation reports and 
questionnaires were administered to the students during their 
regularly scheduled classes. Subjects in each of the four 
education classes were randomly assigned to one of the study's 
four experimental conditions. 
Procedure 

A simulated report was utilized to investigate the 
persuasive effects of source credibility and issue involvement 
on message acceptance. In order to make the report believable, 
a nonprofit company was created. The simulated evaluation 
described a five-year "pilot" study of a career option program. 
The report described the program, objectives, and the 
evaluation results and recommendations. Hypothet ically , the 
five-step career option plan had been implemented in seven 
school districts of a ^•^outhern stote. The report was identical 
for all subjects. Subjects were as' ^ to help evaluate the 
quality of the report before the report was to be distributed 
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to other educators. 

The cover letter contained the experimental manipulation, 
and involved the variables of source credibility and issue 
involvement. The inclusion of the intervention in the cover 
letter was to make the manipulation less obtrusive and to 
minimize react ivity. 

Our operational descriptions for the variables of source 
credibility and issue involvement have been used in several 
previous studies (Maddux & Rogers, 1980; Norton, 1986; Petty, 
Cacioppo, & Heesacker, 1981). These descriptions were adapted 
for the present study. The evaluator titles, "researcher" and 
"art specialist," were determined from previous evaluation use 
research to have high and low evaluator credibility, 
respectively (Braskamp, Brown, & Newman, 1978), In the present 
study the evaluator described as an art specialist was defined 
a^ . speciaixst in the field of music. In the high credibility 
condition the cover letter described the evaluator as a 
professor of educational research. 

Issue involvement, also referred to as personal relevance, 
was manipulated in the cover letter as well. Both involvement 
conditions were designed to not involve direct immediate 
impacts on the subjects. The evaluation report presented to 
the more highly-involved subjects was preceded by information 
that the subject was to help evaluate the quality of the report 
before the report was to be distributed to educators, civic 
organizations, and government officials within Louisiana and 
other southern states. Thus, the possibility of future 
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involvement within the same geographic region was indicated. 

The message for the low involvement group was preceded by 
similar instructions. However, subjects in the low involvement 
condition were told that the results were to be distributed in 
Wisconsin, and that results of the evaluation would not be 
employed for some time. Thus, involvement was portrayed as 
being remote in both time and place. 

Previous empirical research has clearly established that 
stark contrasts of no involvement versus immediate, direct, 
personal involvement do produce discernable differences in 
message processing. But direct immediate involvement is not 
characteristic of real evaluation settings in which decision 
responsibility is often shared and in which movement toward 
decisions is often incremental. Our purpose in designing the 
involvement intervention was to produce a more ecologically 
valid intervention involving a more realistic contrast. Such 
research may also be useful in defining the limen at which 
involvement effects occur. 

An 11-point Likert scale was employed to assess the 
effectiveness of the manipulation of source credibility and 
issue involvement. The same scale has been used in previous 
research (Norton, 1986; petty & Cacioppo, 1979; Pallak & 
Francia, 1985; Swasy & Munch, 1985). The questionnaire 
included 14 items to measure direct manipulation effects, 
divided into the following categories; five items measuring 
credibility, five items to assess adequacy of involvement, and 
the remaining four items in this section acted as fillers used 
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to lessen reactivity. The credibility and involvement items 
were summed to form two overall composite scores that measured 
each dependent variable. Four items in this section were 
reverse phrased to minimize response set. 

Other procedures similar to those utilized in studies by 
Petty and Cacioppo (1979a, p. 1918) and Norton (1986, p. 43) 
were also followed. Thus, subjects were also asked to indicate 
their personal attitude about the topic, i.e., career option 
plans, and their agreement with the evaluator's 
recommendations. 

Results 

Preliminary analyses were performed to test the 
reliability of each of four scales. The scales were: 
credibility, with five items; involvement, with five items; 
attitude, with four items; and five items measuring reaction to 
report recommendations. The alpha reliability coefficients 
were .81 for credibility, .83 for involvement, .88 for 
attitude, and .39 for ecomroendat ions . The low 
internal-consistency reliability for the five recommendation 
items was not unexpected since the items were diverse in their 
content . 

The analysis of the involvement scale indicated a low 
item-total correlation (r « .16) for the first item. Based on 
the result, modifications to that item were deemed necessary 
for the subsequent use of the item. The alpha coefficient for 
the remaining four items, omitting the first item, was .93. 

The study's first research question asked whether 
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peripheral involvement persuasive effects occur even if 
evaluation information does not directly and immediately impact 
evaluation report readers? "Involvement" was measured using 
five 11-point Likert-scale items (e.g., Norton, 1986; Swasy & 
Munch, 1985) with the extreme descriptive anchors, "agree 
strongly" or "disagree strongly", lower scale score;* indicate 
higher involvement. The mean for the high involvement 
condition was 26.39 (SD « 12.93) and the mean for the low 
involvement condition was 28.93 (SD » 12.98). A univariate 
test of these differences was not statistically significant (F 
(1,61) » .65, £ >.05). This finding indicates that the 
subjects did not perceive any difference between the high and 
low involvement conditions. As presented in Table 1, no other 
main or interaction effects were statistically significant 
(alpha = .05) for this dependent variable. Since the scale 
ranged from five (most involved) to 55 (least involved), and 
the means for the two involvement conditions were near the 
middle of this range, the subjects were moderately involved in 
processing the evaluation report information. 

The study's second research question asked, do evaluator 
peripheral credibility cue effects occur independently of 
involvement effects? As noted previously, the subjects rated 
the extent to which they found the source "credible". The 
rating system utilized an 11-point scale where "1" indicated 
"agree strongly" and "11" indicated "disagree strongly." Five 
statements were combined so that maximum acceptance of the 
source equalled five and least acceptance equalled 55. 
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Subjects in the high credibility condition rated the evaluation 
report source as being more credible ( M. = 25.19, SD » 6.96) 
than subjects in the low credibility condition ( W[ « 34.84, SD = 
8.37). This difference in means was statistically significant 
(1,61) « 24.44, £ <.001); the correlation ratio effect size 
for this analysis was 28.6% (1461.24/5108.52). In a 2 x 2 
factorial analysis of variance, presented in Table 2, no other 
univariate null hypotheses involving effects on the credibility 
dependent variable were rejected. 

To address the first two research questions in a 
multivariate context (Fish, 1988), a 2 x 2 MANOVA involving the 
two dependent variables, credibility and involvement, was 
conducted. Only the null hypothesis involving the credibility 
main effect was rejected (lambtia = .76, df « 2/62, £ <..00l). 

Ancillary analyses were performed to test for other 
effects that were also of interest in the study. A two-way 
factorial analysis of variance on attitude toward career option 
plans involved no statistically significant main effects or 
interactions. The findings indicated that the attitudes toward 
career option plans were similar across design conditions. The 
grand mean (13.04; SD « 7.10) indicated a moderately positive 
attitude toward career option plans, since these items were 
rated on a 9-point scale (lower scores being more favorable) 
such that total scale scores therefore ranged from four to 36. 

A second ancillary analysis was computed for the five 
recommendation items. The results indicated that the subjects 
were generally positive (M. - 18.71, SD - 6.77) in their 
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acceptance of the report recommendations. Thus, subjects were 
open to the topic in the evaluation report. 

Discussion 

Conclusions important to the utilization of program 
evaluations may be drawn from the present study. First, 
expectations concerning the detection of source credibility 
information was supported by the findings of this study. The 
findings were in agreement with those of Hovland and Weiss 
(1951) who observed that source credibility is attended to by 
subjects. Also, the source credibility findings were in 
agreement with those in the related program evaluation research 
of Brown, Braskamp, and Newman (1978). 

These findings suggest the importance of attending to 
Standard A2 of the Joint Committee on Standards for Educational 
Evaluation (1981, pp. 24-26). This standard requires that "The 
persons conducting the evaluation should be both trustworthy 
and competent to perform the evaluation, so that their findings 
achieve maximum credibility and acceptance." The results of 
the present study suggest that educators do attend to 
information about evaluator experience and job title when 
making judgments about evaluator credibility. 

The standardized effect size for the credibility 
in*- rvention on perceptions of evaluator credibility was 1.26 
((34.84 - 25.19)/((8.37 + 6.96)/2)). These are very large 
effect sizes. 

Second, the results of the present study indicated that 
the issue-involvement intervention tailed to directly affect 
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perception of involvement, as noted previously and also 
reported in Table 1. This finding is noteworthy given recent 
research which has indicated that issue involvement is an 
important motivational variable in changing how people 
cognitively respond to information (Petty & Cacioppo, 1986; 
Greenwald, 1968). Thus, cognitive response data were collected 
and analyzed in the subsequent study (Eason & Thompson, 1988) 
to further explore this finding. 

More studies emphasizing ecologically valid involvement 
manipulations are needed if we are to understand the subtle 
influences on involvement in a reality in which authority is 
shared and in which change is incremental. What are not n<=*eded 
are more studies offering stark but unrealistic contrasts of 
involvement conditions. 

The results in the present study suggest that evaluator 
credibility is perceived by report readers. Clearly, these 
results suggest that evaluators must actively and 
systematically endeavor to present themselves as being credible 
sources of information. 
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Table 1 

ANOVA Source Table for Involvement 



Effect 



Source 


SOS 




df 




MS 


F 


£ calc 


Si 


ze 


Credibility 


35 


.97 


1 


35 


.97 


.21 


NS 


0 


.4% 


Involvement 


107 


.17 


1 


107 


.17 


.62 


NS 


1 


.0% 


Interaction 


33 


.75 


1 


33 


.75 


.20 


NS 


0 


.3% 


Residual 


10170 


.47 


59 


172 


.'3 










Total 


10349 


.65 


62 


166 


.93 











Table 2 

ANOVA Source Table for Credibility 



















Effect 


Source 


SOS 


df 




MS 


P 


£ calc 


Si 


ze 


Credibility 


1457. 


63 


1 


1457 


.63 


25.38 


.00 


28 


.5% 


Involvement 


5. 


06 


1 


5 


.06 


.09 


NS 


0 


.0% 


Interaction 


253. 


96 


1 


253 


.96 


4.42 


NS 


5 


.0% 


Residual 


3388. 


26 


59 


57 


.43 










Total 


5108. 


52 


62 


82 


.40 











*5 

^8 



